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PREFACE 


I  am  not  a  computer  expert,  but  I  can  see  tne  kind  of 
impact  computers  have  on  our  daily  lives.  The  fact  that  1 
u»rote  this  report  using  a  personal  computer  certainly  had  an 
impact  on  me.  If  made  it  much  more  enjoyable. 

I  am  not  interested  in  computer-based  training  from  a 
technical  point  of  view,  but  rather  from  tne  standpoint  of 
an  instructor.  If  a  computer  can  help  me  teach  better  and 
at  the  same  time  help  a  student  learn  faster,  then  it  would 
seem  to  be  a  tool  worth  using. 

What  this  report  hopes  to  do  is  to  improve  the  way  we 
work  with  computer-based  training  systems  from  an 
organizational  point  of  view.  The  computer  experts  have 
designed  systems  that  can  help  us  teach,  and  now  it  is  up  to 
us  to  look  for  ways  to  use  the  systems  effectively. 
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EXECUTIVE  SUMMARY 


Part  of  our  College  mission  is  distribution  of  the 
students’  problem  solving  products  to  DoD 
sponsors  and  other  interested  agencies  to 
enhance  insight  into  contemporary,  defense 
related  issues.  While  the  College  has  accepted  this 
product  as  meeting  academic  requirements  for 
graduation,  the  views  and  opinions  expressed  or 
implied  are  solely  those  of  the  author  ana  should 
not  be  construed  as  carrying  official  sanction. 


‘insights  into  tomorrow ’ 


REPORT  NUMBER  87-i«i5 

AUTHOR(S)  MAJOR  DONALD  3.  MACNIVEN*  USAF 

TITLE  COMPUTER-BASED  T RAIMI NS  SYSTEMS — ORGANIZING  TO  USE 
THEM 

I.  Pur  pose'.  To  suggest  ways  the  Air  Force  night  organize 
to  no re  effectively  use  computer-based  training  systems. 

II.  Problem:  Although  computer-based  training  <CP.T)  systems 
have  been  shown  to  be  effective  teaching  tools* 
organizational  problems  exist  in  applying  then*  widely  within 
the  Air  Force.  As  CUT  systems  have  become  more  available 
and  affordable*  the  number  of  applications  in  the  Air  Force 
have  expanded.  Today  all  MAJCOMS  are  involved  with  CBT  to 
some  extent.  Organizational  shortfalls  within  the  Air  Force 
have  resulted  in  lack  of  standardization  of  both  hardware 
and  software*  duplication  of  CJT  development  efforts*  and  a 
lack  of  support  and  training  for  CBT  developers.  This 
prevents  the  effective  use  of  CBT. 

III.  Da  fa:  Extensive  developmental  programs  involving  CRT 
within  the  Air  Training  Command  over  many  years  proved  CBT 
worked.  A  wide  variety  of  CBT  systems  is  now  available  and 
trainers  at  the  small  unit  level  are  developing  many  more 
applications-  These  individuals  cite  problems  of  lack  a * 
standardizat ion*  guidance*  and  training. 


_ CONTINUED 

IV.  Conclusion  t  CBT  can  be  effective  if  developed  and 
applied  properly.  The  Air  Force  needs  to  organize  itself  to 
make  better  use  of  the  technology  on  a  wide — scale  basis. 

Net  all  MAJCOMs  have  not  established  effective  single— points 
of  contact  for  C3T  matters.  Add it icmal ly ,  personnel 
involved  with  developing  CBT  are  not  well  supported  or 


trained. 


V.  RecoBwendations?  The  MAJCOMs  must  take  a  more  active 
role  m  monitoring  CBT  applications.  The  Ai'-  Force  should 


establish  an  agency  within  the  Air  Training  Command  with  the 


expertise  and  equipment  to  help  current  and  future  CBT  users 
evaluate  and  develop  effective  programs.  The  Air  Force 
should  develop  a  training  program  for  personnel  involved  in 
CBT  and  track  this  expertise  with  a  special  experience 
i  derv  t  i  f  i  er . 


Vll 
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Chapter  One 


INTRODUCTION 

Computers  have  become  and  will  continue  to  be  an 
integral  part  of  Air  Force  training  programs.  Along  with 
the  other  services  in  the  Department  of  Defense,  the  Air 
Force  has  helped  develop  computer-based  training  (CBT) 
systems  <8: 3)  and  has  shown  them  to  be  effective  in  reducing 
overall  training  costs.  These  systems  use  the  communication 
and  storage  capabilities  of  a  computer  to  provide  direct 
presentation  of  instructional  materials,  allow  for  practice 
by  the  learner  (8:9),  and  also  handle  a  myriad  of 
administrative  tasks  for  the  instructor. 

The  potential  for  CBT  systems  is  vast,  and  every  major 
command  in  the  Air  Porce  is  now  using  the  technology 
<14:  —  > .  New  advances  in  computer  hardware  and  software 
will,  in  the  near  future,  enable  the  systems  to  run  on 
desktop  computers  and  be  coupled  with  videodisc  players, 
making  CBT  even  more  available  at  the  local  level.  The 
attractiveness  of  the  technology  does  not  however,  lie  with 
its  "bells  and  whistles"  approach  but  with  its  effectiveness 
as  a  teaching  tool  and  the  cost  benefits  it  can  provide. 
While  CBT  systems  can  produce  powerful  instructional  tools, 
the  flurry  of  development  activity  by  government  agencies 
and  private  industry  aiixe  has  not  been  without  its 
proDiems . 


STATEMENT  OF  THE  PROBLEM 

Currently,  the  Air  Force  is  not  well  organized  to  handle 
CBT  development  and  realize  aLi  of  the  training  benefits  it 
has  to  offer.  Studies  of  CBT  systems  have  shown  that 
computers  can  increase  training  effectiveness,  reduce  total 
training  Lime,  and  lighten  administrative  workloads  (9:3-6). 
As  a  result  of  these  potential  savings,  the  Air  Force  made  a 
commitment  to  use  computers  in  training  and  now  has  a  vast 
array  of  systems  in  use.  This  commitment  to  the  technology 
however,  has  not  been  followed  by  the  necessary 
organizational  changes  needed  to  manage  CBT  effectively. 
Today,  both  present  and  potential  users  of  CBT  are  faced 
with  a  lack  of  standardization  in  equipment  and  computer 
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languages,  little  or  no  training  for  themselves,  and  no 
place  to  go  for  help.  This  has  resulted  in  unnecessary 
duplication  of  program  development  efforts,  a  lack  of 
coordination  and  communication  among  CBT  users,  and  an 
overall  patchwork  approach  to  the  application  of  CBT 
systems.  This  is  not  an  effective  use  of  valuable  training 
dollars.  Training  costs  have  always  consumed  a  huge  part  or 
the  Air  Force  budget  in  terms  of  manpower,  money,  and 
equipment;  and  at  a  time  when  sharp  budget  cuts  are  eminent, 
the  Air  Force  must  make  the  most  out  of  every  training 
dollar  it  spends.  Accordingly,  the  Air  Force  needs  to  make 
organizational  changes  in  order  to  use  CBT  to  its  full 
advantage . 


ASSUMPTIONS 

Computer -assi sted  instruction  (CAI)  is  defined  as  the 
use  of  the  communication  and  storage  capabilities  of  a 
computer  to  manage  the  direct  presentation  of  instructional 
materials  and/or  provision  of  practice  to  the  learner  (8:9). 

Computer-managed  instruction  (CMI)  is  the  use  of  a 
computer  for  testing,  scheduling,  allocating  resources, 
collecting  student  data,  and  providing  status  reports 
(8:13). 

Computer-based  training  (CBT)  systems  use  a  combination 
of  both  CAI  and  CMI. 

Computer  based-training  systems  (CRTS)  are  not 
applicable  to  every  training  situation.  As  an  example,  they 
would  not  be  cost  effective  if  used  with  a  very  smaii 
student  population  or  for  one-time  training  requirements 
that  could  be  presented  by  lecture  or  in  print.  CBT  systems 
should  be  integrated  where  needed  as  part  of  an  overall 
systems  approach  to  training  requirements. 

Uhen  properly  designed  and  used,  CBTS  can  produce 
quality,  cost  effective  training . 


LIMITATIONS 

The  use  of  simulators  and  computerized  gaming  is  not 
within  the  scope  of  the  study. 


Chapter  Two 


HISTORY  OP  COMPUTER-BASED  TRAINING  SYSTEMS 
IN  THE  AIR  FORCE 


Educators  have  been  interested  in  computers  as 
instructional  tools  for  decades  (3:7).  They  saw  programmed 
texts,  developed  in  the  late  1950s  and  early  1960s,  as  ideal 
candidates  to  convert  to  computer  instruction.  Eventually, 
educators,  trainers,  and  behavioral  scientists  adapted  the 
programmed  texts  for  computer  presentation  as  computers 
became  more  readily  available.  The  instruction  took  the 
form  of  a  traditional  text  broken  down  into  'frames'. 
Basically,  the  computer  presented  the  'frames',  asked 
questions,  and  responded  to  multiple  choice  answers  from  the 
student.  [f  the  student's  response  was  correct,  the 
computer  displayed  the  next  'frame*.  If  the  student 
answered  incorrectly,  the  computer  presented  the  material 
again.  During  these  first  attempts  at  computer-aided 
instruction  (CAI),  the  early  computers  were  expensive  to 
buy,  maintain,  and  program.  As  a  result,  program  developers 
saw  CAI 's  future  to  be  with  large-scale  systems  "using  many 
students  to  amortize  the  costs"  (3:7).  Since  the  military 
had  an  ongoing  need  for  training  large  numbers  of  students, 
the  services  were  an  ideal  testing  ground  for  CAI. 

The  Air  Force  helped  pioneer  the  development  of  CAI 
with  projects  occurring  at  Chanute,  Sheppard,  and  Lowry 
Technical  Training  Centers  (8:3).  The  PLATO  (Programmed 
Logic  for  Automated  Teaching  Operations)  computer-based 
instruction  system  was  used  extensively  at  Chanute  and 
Sheppard  Technical  Training  Centers  while  the  Advanced 
Instructional  System  (AIS)  was  tested  at  Lowry  APB. 


PLATO 

PLATO  started  as  a  research  project  at  the  University 
of  Illinois  in  1959  (2:33)  and  is  now  the  oldest  and  largest 
computer-based  instruction  system  available  (8:3).  In  1972, 
the  Air  Porce  began  a  tri-service  evaluation  of  PLATO  after 
receiving  funding  from  the  Defense  Advanced  Research 
Projects  Agency.  The  Technical  Training  Center  at  Chanute 
AFB ,  Illinois  was  chosen  to  perform  the  evaluation  with  the 


first  phase  beginning  in  July  1972.  This  phase  continued 
until  June  1974  and  included  the  operations  pianninq  and 
acquisition  of  equipment.  During  the  second  phase  (June 
1974  until  January  1976)  Air  Force  members  working  close  i  y 
with  civilian  contractors  developed  CAT  to  be  part  of  the 
Special  Purpose  Vehicle  Repairman  courses  at  Chanute.  The 
third  phase  began  in  October  1975  and  ran  until  September 
1976  and  consisted  of  a  series  of  seven  experimental  lessons 
(8:3).  While  not  without  its  problems,  the  entire  program 
produced  dramatic  results  and  was  an  overwhelming  success. 

In  the  area  of  instructional  effectiveness,  student 
test  scores  showed  the  PLATO  program  was  as  effective  as  the 
regular  course  with  a  28%  savings  in  time.  Instructor 
evaluations  noted  students  using  PLATO  were  more  attentive 
when  compared  to  those  using  programmed  texts  and  workbooks 
(8:4). 

With  the  computer  technology  of  the  mid  1970s,  the 
PLATO  program  was  more  costly  than  the  conventional  course 
by  587,500.  "However,  cost-avoidance  savings  due  to  the 
total  combination  of  instructional  system  development 
procedures,  PLATO,  and  instructional  material  design  was 
determined  to  be  5180,000  a  year”  (8:4). 

Also  of  concern  to  evaluators  of  the  project  was  the 
impact  PLATO  had  on  students*  and  instructors'  attitudes. 
Course  critiques  from  students  using  PLATO  showed  they  had  a 
more  positive  attitude  toward  the  course  than  other  students 
not  using  the  PLATO  instruction.  Instructors  attitudes  were 
initially  positive,  but  declined  as  they  became  dissatislied 
with  their  less  direct  role  in  the  instruction  process.  It 
became  clear  that  the  changing  role  of  the  instructor  should 
be  a  concern  for  future  CBT  systems  (8:4). 

Throughout  the  project  it  became  apparent  that  course 
instructors  by  themselves  could  not  develop  large-scale  CAI . 
Evaluators  felt  a  ream  approach  drawing  on  the  expertise  of 
computer  programmers,  systems  designers,  and  subject-matter 
experts  was  needed  to  produce  the  best  computer-aided 
instruction  (8:4). 

Overall  positive  results  achieved  at  Chanute  were  also 
seen  in  tests  of  PLATO  at  the  Air  Force's  School  of  Health 
Care  Sciences.  The  school,  located  at  Sheppard  APB.  Texas, 
ran  an  evaluation  of  PLATO  between  1975  and  1977  (5:9). 
Applied  to  the  Physicians  Assistant  course,  CAI  showed 
encouraging  results  in  both  student  performance  and  reduced 
training  time. 


Hit.h  CAI,  student  performance  increased  by  18%  over 
traditional  lecture  based  instruction.  For  low  aptitude 
students,  it  proved  to  be  17%  more  efficient  than  programmed 
texts.  Additionally,  high  aptitude  students  showed  a  time 
savings  of  from  29-32%  (8:5).  In  student  course  critiques, 
"favorable  comments  toward  CAI  were  primarily  related  to  the 
ability  to  progress  through  the  course  on  a  self-paced 
basis"  (5:20). 

In  the  area  of  management  however,  "the  results  of  the 
PLATO  evaluation  at  Sheppard  APB  pointed  out  the  need  for 
extensive  preplanning  activities  to  ensure  that  appropriate 
staff  personnel  are  in  place  and  that  course  instructional 
staff  are  trained  prior  to  the  implementation  of  CAI"  (8:5) 


AIS 

The  Advanced  Instructional  System  (AIS)  "was  developed 
to  demonstrate  the  feasibility  of  managing  and  administering 
individualized  instruction  for  up  to  2,000  students  daily  in 
four  Air  Porce  technical  training  courses”  (9:i.)  This 
computer-aided  instruction  system  "was  to  be  the  first  large 
scale  integration  of  technology  to  produce  cost-effective 
individualized  instruction”  (7:3).  It  was  designed  to 
provide  a  "full  range  of  Computer-based  Instruction  (CBI) 
functions,  including  course  development  and  presentafc ion , 
resource  allocation  and  scheduling,  and  individual  student, 
management  (9:i).  Presently,  AIS  is  being  used  in  a  variety 
of  programs  within  the  Air  Force  Systems  Command,  the 
Strategic  Air  Command,  and  the  Tactical  Air  Command  (8:5). 

An  important  part  of  the  AIS  program  was  its  use  of  an 
authoring  system  designed  to  make  it  easier  for  instructors 
to  write  CAI  without  extensive  training  in  computer 
programming.  Based  on  experience  with  the  PLATO  system, 
recommendations  for  future  computer-aided  instruction 
programs  called  for  the  use  of  "a  development  team 
consisting  of  both  instructors  and  CAI  authors"  (8:6).  AIS 
simplified  the  authoring  system  so  the  subject  matter 
experts  (instructors)  could  also  produce  the  CAI  material 
without  being  experienced  computer  programmers.  The  results 
from  AIS  showed  that  instructors  could  develop  effective  CAI 
materials  when  using  the  authoring  system  and  that  the 
availability  of  a  simplified  authoring  system  had  a  great 
deal  of  impact  on  the  organizational  acceptance  of 
computer-aided  instruction  (8:6). 

AIS  produced  very  positive  results  from  the  beginning 
of  its  operation  in  1975  through  the  end  of  the  evaluation 
in  1979.  First,  the  system  averaged  11.2%  less 


administrative  tine.  Also  A1S  lessons  had  only  a  61  failure 
rate  compared  to  an  average  23%  failure  rate  with  prograaaed 
texts  <8:2>.  And  20,000  students  graduated  f roa  AIS  courses 
with  demonstrated  training  time  reductions  of  15-501.  Their 
overall  attitudes  were  positive  with  80-901  reacting 
favorably  to  the  instruction.  Test  achievement  among  the 
students  was  comparable  to  conventional  instruction.  Six 
months  after  the  students  completed  training,  follow-up 
questionnaires  sent  to  their  supervisors  rated  951  of  the 
students  as  satisfactory  or  better.  The  majority  of  the 
students  (681)  were  rated  in  the  two  highest  categories, 
either  'very  satisfactory'  or  'excellent'  <9:5>.  Clearly 
the  AIS  evaluation  demonstrated  the  capability  of  CBI  to 
produce  large  numbers  of  highly  qualified  students  while 
doing  it  in  less  tine  than  conventional  methods. 

Both  the  PLATO  and  the  AIS  systems  required  large 
mainframe  computers  and  an  extensive  support  team  to 
develop,  program,  and  administer  CAI  and  CHI;  but  they 
proved  the  computer  to  be  an  effective  teaching  tool.  They 
helped  pave  the  way  for  future  refinements  in  computer-based 
training  systems  and  helped  spark  the  imagination  of 
trainers  concerning  new  applications  of  this  technology.  As 
a  result  of  its  experience  with  these  two  systems,  the  Air 
Force  decided  to  use  CBT  more  extensively. 


Chapter  Three 
PRESENT  CBTS  ACTIVITIES 


The  Air  Force  and  the  Department  of  Defense  have  made  a 
large-scale  commitment  to  computer-based  traininq.  It  is 
easy  to  understand  why  when  one  views  the  magnitude  of 
training  requirements.  The  Air  Traininq  Command  alone 
annually  "trains  344,000  people  in  more  than  4,300  courses 
covering  some  300  specialties”  (1:66).  Since  CBT  can 
produce  results  equal  to  or  better  than  conventional  methods 
and  sxqnit icant ly  reduce  the  training  time  required,  there 
is  little  wonder  why  CBT  systems  are  so  appealing. 

Come  of  the  current  Air  Porre  computer  based  traininq 
systems  efforts  are  listed  below 

ATS.  The  Advanced  Traininq  System  is  designed  to  provide 
the  Air  Traininq  Command  a  computer-based  traininq  system 
with  a  wide  range  of  capabilities.  The  system  accomplishes 
"six  training  delivery  tasks:  information  presentation, 
demonstration,  drill  and  practice,  evaluation,  feedback,  and 
remediation”  (8:6).  The  management  function  schedules  the 
students,  instructors,  and  equipment.  It  also  follows  paeh 
student's  progress  and  evaluates  the  instructional  process. 

BLTHS.  The  Base  Level  Training  Ranagement  System  is  an 
in-house  effort  of  the  Air  Training  Command.  The  system 
provides  "a  computer  based  data  collection  and  management 
capability  that  provides  an  interface  between  ATC  technical 
courses  and  the  Advanced  Personnel  Data  System"  <8:6> 
Additionally,  it  provides  course  management  and  course- 
documentation. 

--TRIM.  The  Time  Related  Instructional  Ranagement  system 
provides  CAI  for  Undergraduate  Pilot  Training,  plus  t light 
training  management  and  scheduling  support  (8:6) 

--PLATO.  CAI  continues  to  be  provided  by  the  PLATO  system 
at  Sheppard  and  Chanute  Technical  Training  Centers  *8:/) 
Contract  instruction  using  the  PLATO  system  is  a : so  used  in 
programs  such  as  KC  ID  aircrew  training  (/>:36). 


-CDTS.  The  Coaputer  Directed  Training  System  originated  at 
Keesier  APB,  Mississippi  in  1968  as  a  way  of  using  a 
coaputer  to  teach  people  how  to  use  a  computer.  It  provides 
on-the-job  training  for  the  Base  Level  Military  Personnel 
System  and  the  Horid  Hide  Military  Command  and  Control 
System  ( 8 : 7) . 

-ISS.  The  Instructional  Support  System  provides  software 
for  supporting  computer-aided  instruction  and 
computer-managed  instruction  and  is  available  for  Doll  use. 
The  software  is  designed  to  operate  on  a  variety  of  computer 
systems  and  was  developed  from  the  AIS.  It  uses  Ada,  a 
computer  language  designed  to  be  compatible  throughout  the 
DoO  (8:7). 

--AOTS.  The  Advanced  On-the-Job  Training  System  is  under 
development  at  Bergstrom  APB,  Texas.  It  is  designed  to  make 
base  level  OJT  more  effective  by  giving  supervisors  a 
training  plan  individually  tailored  to  each  worker.  It 
compares  job  requirements  with  the  worker's  qualifications 
and  produces  a  prioritized  OJT  schedule.  "ACTS  may  ident  ly 
and  schedule  hy  task,  qualified  and  available  trainers, 
technical  references  study  guides,  supplies,  equipment, 
weapons  systems,  and  facilities  necessary  to  support 
t  ra  i  n  mg'*  (1:68). 

TRIADS.  This  is  a  Department  ot  Defense  joint  service 
effort  that  involves  developing  a  library  of  programs  that 
relate  to  computer  based  instruction.  The  purpose  is  to 
"develop  software  quality  standards  and  instructional 
quality  standards  for  programs  to  be  included  in  the  system 
library,  demonstrate  the  programs  and  develop  user  training” 
"The  hardware  for  TRIADS  is  to  feature  modularity 
and  mterdeviee  eompat i bi l i ty”  and  the  software  is  to  be 
based  on  Ada,  the  new  Dot)  language  (8:8). 

TPS5T.  The  Training  and  Periormance  Support  System  is  a 
computer  based  system  developed  for  the  Air  Porcc  Systems 
Command  ’The  system  was  designed  for  nov i ce  and  less 
experienced  acquisition  managers  and  provides.,  individual/ 
tailored  curriculum  .  ‘as  rpguired'.  Karh  lesson 
topic,  .provides  job  performance  aids  in  the  foim  of  policy, 
lessons  learned,  procedures,  and  technical  background 
information”  (8:8) 

As  you  can  see,  a  wide  var.ety  of  programs  are 
underway,  but  they  an*  not  without  significant  problems 
Mosf  of  *  he  systems  r  *  *  I y  on  different  computers  with 
different  software  The  Dof)  nas  recognized  the  problem  of 
lack  of  st  and.ird, /at  ion  m  software  and  has  developed  Ada, 
designed  to  he  a  standard  language,  but  it  will  he  many 


years  before  all  existing  programs  are  converted,  if  at  all. 
A  standard  for  computers  has  yet  to  be  established.  The 
computer  industry  is  developing  so  rapidly  that  systems 
become  outdated  quickly.  Recent  advances  in  commercially 
developed  CDT  software  has  made  courseware  preparation 
easier  and  reduced  the  number  of  manhours  needed  to  produce 
each  hour  of  computer  instruction.  This  is  both  good  and 
bad.  It  makes  CBT  more  available  on  a  smaller  scale,  but  at 
the  same  time  it  makes  standa rdi 2ation  that  much  harder. 
ALong  with  the  problem  of  standardization  is  one  of 
training.  The  early  tests  involving  PLATO  and  A1S  pointed 
out  the  need  for  instructors  trained  in  the  use  of  CBT  and  a 
team  approach  to  course  development.  New  systems  boast  of 
easy  to  prepare  CBT  but  experience  is  proving  that  new 
course  developers  need  good  training  to  produce  quality 
materials . 

Changes  are  coming  in  how  the  DoD  is  organizing  to  make 
better  use  of  CBT  and  are  covered  in  the  next  chapter. 


Chapter  Pour 


ORGANIZATIONAL  CHANGES  WITHIN  THP.  DoD 

After  years  of  study  and  analysis,  aany  of  the 
technical  problem;  initially  encountered  with  computer  based 
training  systems  have  been  solved.  The  question  today  is 
not  ’’Should  ue  use  computers?”.  but  "How  can  we  best  use 
computers?"  The  systems  have  proven  useful,  and  the 
coaaercial  marketplace  now  has  a  ayriad  of  coaputers  and 
programs  to  produce  computer  based  training  The  Department 
uf  Defense  helped  pioneer  this  effort  with  programs  in  each 
of  the  services.  To  reduce  duplication  of  effort  and  to 
standardize  some  of  the  equipment  and  software  used  in  the 
buD ,  several  key  orgam zat i ona 1  and  aanaqeaent  changes  have 
taken  place  in  recent  years.  In  1982,  the  Report  of  the 
Defense  Science  Board  on  Training  and  Training  Technology 
concluded  that  "major  improvements  in  training  are  necessary 
and  that  ’echnology  will  contribute  s i on i f i cant  1 y  coward 
effecting  these  improvements”  (I0:iii)  A«ong  the  primary 
conclusions  of  the  study  was  one  which  stated  "Huch 
improvement  in  readiness  is  available  through  improved 
training  High  technology  can  help.  Rapid  progress  is 
possible  with  promise  of  high  payoff  by  funding  known 
successful  app 1 i cat i ons  such  as  computer  aided  instruction” 
(10  :v)  But  ,  t.  also  went  on  to  say,  "Easily  identifiable 
proponency  is  missing  in  OSD  and  the  Military  Departments  to 
direct  R*0  relating  to  training,  to  review  technology  for 
training  applications  and  lo  influence  training  initiatives” 
<10:v).  In  other  words,  there  is  a  lack  of  central  locus 
and  cnnt rol 

The  Defense  Science  Board  panel  made  some  key 
recommendations  in  its  November  l 98?  report  The  Secretary 
of  Defense  reviewed  their  findings  and  approved  them  in 
February  1989  The  first  recommendation  under  organization 
and  management  (one  of  three  areas  addressed)  was  to 
establish  a  steering  committee  focusing  on  traimnq  and 
training  technology.  The  fhatraan  of  the  Join)  ’’hiefs  of 
Staff  has  initiated  this  act-,  on  *o  get  ”a  high  level 
perspective  and  proponencv”  on  the  overall  training  system. 
There  was  a  need  to  "strengthen  the  position  >f  training  at 
the  hudqet  table,  and  help  to  prevent  administrative  and 
technical  duplication  of  effort” 

lo 


The  next  recommendat ion  was  to  establish  a  "Defense 
Training  Data  Center  for  all  trai ning- related  data"  (10:6). 
The  Defense  Training  Data  and  Analysis  Center  <TDAC)  was 
established  in  Orlando,  Plorida  (the  name  has  since  been 
changed  to  the  Training  Performance  and  Data  Analysis 
Collection,  <i6:  -)>  and  is  collocated  with  the  Naval 
Training  Equipment  Center  and  the  U.S.  Army  Program  Manager, 
Training  Devices  (4:47).  The  center  is  the  focal  point 
within  the  Office  of  the  Secretary  of  Defense  for  defense 
t raininq- related  data  such  as  training  loads,  flow  rates  and 
training  support.  Their  chief  "product  is  information 
needed  lor  tieci  s  1  on- malt  1  ng  .  ”  (4:47) 

The  third  recommendation  was  to  "revise  the  acquisition 
process  to  (a)  ease  procurement  speci f i ca t ions  and 
standards,  commensurate  with  training/device  use;  (f>) 
acquire  training  requirements  data  earlier  in  the  weapon 
system  development  cycle”  <10:8>  Many  training  devices 
were  being  needlessly  "over  engineered"  when  they  were 
designed  to  meet  standards  for  equipment  intended  for  field 
and  combat  conditions.  There  was  also  a  frequent  complaint 
that  training  devices  arrived  too  long  after  l he  weapons 
systems  were  fielded  <!0:8).  The  recommendation  was  made  to 
save  time  and  money  by  "eliminating  excessive 
design/manuf acturtng  requirements"  and  "to  have  the  training 
devices  in  place  hy  the  time  they  are  needed”  <10:8). 

Another  recommendation  was  to  have  the  "service 
laboratories  increase  fundinq  and  management  emphasis  on 
training  technology”  (10:9).  The  board  felt  that  "with  few 
notable  exceptions,  the  laboratories'  RAD  priorities  are  not 
driven  by  operational  requirements  cr  problems.  Moreover, 
operational  people,  when  fared  with  immediate  applied 
questions,  rarely  look  to  the  I aboralor i es  for  answers" 
(10:9). 

The  final  recommenda t i on  in  organization  and  management 
was  to  "provide  a  single  point  of  contact  for  proponency  anil 
coordination  of  training  and  training  technology”  within 
each  service  (10:9)  This  was  to  help  commun i cat  ion  and 
make  the  best  use  of  limited  resources.  The  Air  Porce  now 
has  an  office  which  serves  as  its  single  point  of  contact 
and  it  is  HQ  USAP/XOfTTi  <14:  > 

The  process  of  integrating  computer  based  training 
systems  has  in  the  author  s  opinion  taken  two  giant  sieps 
with  a  third  yet  to  be  comp  1  ei  eri .  :’t  op  one  was  the  design, 

development  and  testing  of  the  technology  Step  two  was  the 
recognition  and  action  at  the  DoD  level  to  organ. ze  the 
defense  training  community  to  establish  i  ng  l  e  points  of 
coni  act  within  earh  service  for  training  technology  needs. 


to  standardize  systems,  and  to  provide  for  information 
sharing.  Ail  these  efforts  help  make  the  training  community 
better  able  to  state  its  case,  operate  more  efficiently  and 
ultimately  provide  better  and  cheaper  training.  The 
initiatives  have  yet  to  stand  the  test  of  time,  but  they 
represent  a  good  start.  The  third  step  is  for  the  Air  Force 
to  organize  itself  to  use  CBT  better. 


Chapter  Five 


CBT  ORGANIZATIONAL  PROBLEPIS  IN  THE  AIR  FORCE 

The  Air  Force  is  not  well  organized  to  support  CBT 
within  snail  organizations.  The  Air  Training  Command  has  a 
great  deal  of  experience  working  with  CBT  on  a  large-scale, 
but  experience  is  lacking  within  many  smaller  organizations. 
Hueh  of  the  present  CBT  development  is  occurring  in  small 
units  such  as  combat  crew  training  squadrons,  the  Air 
Command  and  Staff  College,  and  individual  departments  within 
the  faculty  of  tne  Air  Force  Academy.  To  assist  CBT  in  the 
areas  it  is  growing  the  fastest,  the  smaller  units,  the  Air 
Force  must  do  several  things. 

Pit st,  the  Air  Force  needs  single  points  of  contact  for 
CBT  matters  at  each  of  the  MAJCOHs.  At  present,  not  all 
I? AJCOHS  have  established  an  office  of  primary  responsibility 
< OPR)  and  this  is  causing  confusion  and  wasted  effort  at 
many  levels.  HQ  USAP/XOOTH  (the  Air  Force's  CBT  OPR)  held  a 
conference  in  December  1986  and  discussed  computer-based 
training  systems.  -  Uhile  each  of  the  NAJCOHs  was 
represented,  the  feeling  was  the  Air  Force  as  a  whole  was 
’’not  very  well  organized"  <14:  —  >  to  use  the  systems  to 
their  greatest  advantage.  In  one  command,  the  DO,  LG,  and 
IN  communities  were  all  developing  and  using  CBTS  without 
knowledge  of  what  the  others  were  doing.  When  problems  of 
lack  of  coordination  were  discovered  in  another  command's 
aircrew  training  community,  the  policy  of  NAJCOH/DO  checkotf 
was  established  to  force  coordination  at  least  to  that  level 
(  H  -  )  . 

The  situation  is  particularly  frustrating  for  those 
units  that  can  see  benefits  from  CBT  and  want  to  use  thp 
available  technology.  A  great  deai  of  time  is  spent  jus' 
getting  started.  With  no  central  point  of  contact  ,  the 
potential  user  is  forced  to  develop  his  own  sources  of 
information  on  systems  capabilities,  availability, 
development,  and  application  The  423bth  Strategic  Training 
Squadron  has  become  the  advocate  oi  computer  based  training 
for  the  operations  field  within  the  Strategic  Air  Command. 
They  found  "one  shortcoming  appeared  and  reappeared  again 
and  again:  Organization.  There  weren't  any  recuqni  ::ab  l  e 

CBT  advocates  around. ..  there  were  CBT  supporters  but  they 


weren't  out  there,  day  in  and  day  out,  advocating  CBT 
applications.”  They  took  their  concerns  to  SAC  headquarters 
and  within  three  Months  were  appointed  the  CBT  OPR  for  SAC 
Operations  (12:1-4). 

It  is  easy  to  imagine  how  potential  CBT  users  could  end 
up  duplicating  the  mistakes  of  others.  Dr.  Jerry  Boling  of 
the  International  Officer  School  at  Maxwell  APB,  Alabama, 
stated  there  is  a  need  for  information  crossflow.  There  is 
a  need  to  know  what  systems  are  available,  who  is  doing 
what,  and  what  are  the  ingredients  for  a  successful  program 
(15:--).  An  effort  is  currently  underway  by  Major  Mike 
HcKi*  (a  Reservist  assigned  to  Air  University)  to  produce  a 
newsletter  to  aid  communication  in  what  has  become  a  word  of 
mouth  community.  He  has  undertaken  this  effort  on  his  own 
including  personally  shouldering  ail  printing  costs  <17: — >. 
A  command  single-point  of  contact  could  go  a  long  way  to 
meet  the  information  crossflow  needs  involving  CBT  and  make 
individual  efforts  unnecessary. 

Air  University  has  taken  the  first  step  to  providing 
that  contact  within  its  command.  AU/XPZ  is  responsible  for 
training  technology,  and  they  have  established  the  Air 
University  Technology  Applications  Committee  Lo  increase  the 
sharing  of  ideas  and  resources  between  the  component,  schools 
of  the  Command  (6 : vii > .  This  structure  is  not  without  its 
problems,  but  it  is  responding  to  recognized  needs. 

In  addition  to  MAJCOH  single-points  of  contact,  the  Air 
Porce  should  establish  an  organization  with  the  ability  to 
help  users  develop  effective  computer-based  training  systems 
quickly  and  easily.  Currently,  many  of  the  user  developed 
training  proqrams  are  produced  through  trial  and  error. 
Individuals  nave  had  to  train  themselves  on  the  use  of  the 
computer  systems,  a  task  that  has  not  proved  easy.  The  Air 
Command  and  Staff  College  Directorate  of  Associate  Programs 
had  problems  using  an  authoring  system  when  producing'  their 
own  computer-based  instruction.  They  found  the  curriculm 
writers,  who  were  subject  matter  experts  not  computer 
programmers,  could  not  immediately  use  the  system.  The 
designers  of  the  system  assumed  that  all  of  the  educational 
questions  had  been  answered  and  the  lesson  logic  had  been 
developed,  and  ail  of  the  Instructional  Systems  Development 
concerns  had  been  taken  into  account.  Thus,  ACSC  found  the 
authoring  system  reduced  the  need  for  a  computer  programer, 
but  a  systems  designer  was  still  needed  to  work  closely  with 
the  subject-matter  experts  (11:1-20). 

The  Air  Porce  Academy  has  developed  a  Computer  Learning 
Laboratory  <CLL)  to  solve  problems  at  the  Academy  that  are 
similar  to  those  encountered  by  ACSC.  The  Academy  describes 


the  CLL  as  "an  emerging  concept  of  service  to  the  faculty 
and  students."  Its  purpose  Is  "to  enhance  undergraduate 
education  through  the  use  of  computer-based  tools" 

(13:1-86).  The  CLL  helps  the  various  faculty  departments 
prepare  computer-based  instruction  by  providing  the  computer 
expertise  uith  courseware  development.  The  various 
departments  supply  subject-matter  experts,  and  the  CLL 
advises  on  the  appropriateness  of  computer-based  tools, 
offers  the  programming  expertise,  provides  computers,  and 
helps  develop  the  best  product  for  the  need.  The  CLL  also 
experiments  with  new  technology  products  looking  for  ways  to 
apply  them  to  education. 

An  Air  Porce-uide  resource  such  as  the  Academy's  CLL 
could  help  immeasurably  in  developing  computer-based 
training.  The  4236th  Strategic  Training  Squadron's 
Technical  Applications  Branch  at  Carswell  APB,  Texas,  is  one 
unit  that  has  been  forced  to  get  much  of  its  computer-based 
training  experience  the  hard  way.  They  had  an  opportunity 
to  develop  a  CBT  system  for  the  PB-lil  Avionics 
Modernization  Program  when  they  were  tasked  to  produce 
flight  crew  training  devices  to  teach  the  avionics  updates. 
They  realized  "that  there  were  no  existing  guidelines  for 
systematically  designing,  selecting,  and  implementing  a 
fully  operational  CBT  system  for  military  appl ications . " 

They  used  a  team  approach  and  developed  a  checklist  which 
enabled  them  to  accomplish  their  task.  They  offer  these 
words  of  caution  to  others:  "CBT  is  one  medium  of 
instruction  and  should  be  chosen  as  carefully  as  another 
medium  in  accordance  uith  the  I SD  model.  It  can  be  a 
superior  method  of  instruction  if  used  properly,  but  used 
improperly,  it  can  be  more  like  an  albatross”  (12:1-2).  If 
an  agency  like  the  Academy's  CLL  had  been  available  to 
assist  in  CBT  development,  the  task  would  have  been  much 
easier  for  the  4235th  STS.  Many  others  could  avoid  a 
potential  "albatross"  as  well. 

The  Air  Porce  also  needs  to  develop  a  training  program 
for  those  involved  uith  computer-based  training.  It  seems 
obvious  that  people  developing  computer-based  training  for 
others  need  to  know  what  they  are  doing,  but  to  date  much  of 
that  experience  has  been  gained  through  trial  and  error. 
Members  of  the  4235th  STS  have  been  directly  involved  with 
CBT  development  for  over  four  years  but  readily  admit  that 
the  experience  was  gained  through,  "years  of  trial  and 
error,  learning  to  ask  the  correct  questions  and  of 
developing  successful  solutions  to  the  various  training 
tasks  we  were  tasked  to  support"  (12:1-2).  Others 
illustrate  this  situation  when  they  tell  of  a  lack  of 
expertise  with  CBT  throughout  the  Air  Porce.  In  one 
instance,  a  complete  interactive  videodisc  system  was  found 


in  a  hospital  storage  room.  The  system  had  been  procured 
through  the  efforts  of  a  CBT  advocate  on  the  hospital 
training  staff,  but  after  that  individual  was  transferred, 
no  one  else  knew  how  to  use  it  (IB: — ). 

Some  might  argue  that  the  Air  Force  should  get  out  of 
the  computer-based  training  business  and  that  contract 
support  is  the  way  to  go  but  there  will  always  be  a  need  for 
sub ject-malter  experts  to  assist  in  program  development. 

Prom  the  first  Air  Force  experience  with  PLATO,  program 
evaluators  recoaaended  the  need  for  a  team  approach  to  CBT 
development.  It  makes  good  sense  to  train  Air  Force  members 
responsible  for  producing  quality  CBT  products.  Once 
trained,  members  could  receive  a  special  experience 
identifier  (SET)  that  would  be  used  to  track  them  in  the 
personnel  management  system.  No  longer  would  that  valuable 
experience  be  "lost”  and  not  able  to  be  drawn  upon  at  a 
later  date  if  need  be. 

The  Air  Force  could  improve  the  way  it  is  organized  to 
handle  CBT  by  acting  on  the  recommendations  in  the  next 
chapter . 
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Chapter  Six 


RECOMMENDATIONS 

1.  The  Air  Force  should  establish  single-points  of  contact 
for  computer-based  training  within  each  of  the  MAJCOMs. 

These  single-points  of  contact  should  monitor  and  assist 
computer-based  training  efforts  within  their  command  and 
maintain  close  liaison  with  the  Air  Porce  OPR. 

2.  The  Air  Force  should  establish  an  agency  within  the  Air 
Training  Command  with  the  expertise  and  equipment  to  aid 
current  and  future  computer-based  training  users  develop 
effective  programs.  The  agency  should  be  prepared  to 
provide  computer  expertise  and  work  in  concert  with 
subject-matter  experts  to  prepare  computer-based  instruction 
appropriate  for  the  need.  The  Air  Force  Academy's  Computer 
Learning  Laboratory  could  serve  a  model  for  this  new 
organization . 

3.  The  Air  Force  should  develop  a  training  program  within 
the  Ait  Training  Command  for  personnel  involved  in 
developing  computer-based  training  programs.  A  special 
experience  identifier  <SEI>  should  be  used  to  track 
personnel  trained  and  experienced  with  computer-based 
training  systems. 
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